npj Digital Medicine

BackgroundLongitudinal tracking of multiple sclerosis (MS) symptoms in an individuals own environment may improve self-monitoring and clinical management for people with MS (pwMS). ObjectiveWe present a machine learning approach that enables longitudinal monitoring of clinically relevant patient-reported symptoms for pwMS by harnessing passively collected data from sensors in smartphones and fitness trackers. MethodsWe divide the collected data into discrete periods for each patient. For each prediction period, we first extract patient-level behavioral features from the current period (action features) and the previous period (context features). Then, we apply a machine learning (ML) approach based on Support Vector Machine with Radial Bias Function Kernel and AdaBoost to predict the presence of depressive symptoms (every two weeks) and high global MS symptom burden, severe fatigue, and poor sleep quality (every four weeks). ResultsBetween November 16, 2019, and January 24, 2021, 104 pwMS (84.6% women, 93.3% non-Hispanic White, 44.0{+/-}11.8 years mean{+/-}SD age) from a clinic-based MS cohort completed 12-weeks of data collection, including a subset of 44 pwMS (88.6% women, 95.5% non-Hispanic White, 45.7{+/-}11.2 years) who completed 24-weeks of data collection. In total, we collected approximately 12,500 days of passive sensor and behavioral health data from the participants. Among the best-performing models with the least sensor data requirement, ML algorithm predicts depressive symptoms with an accuracy of 80.6% (35.5% improvement over baseline; F1-score: 0.76), high global MS symptom burden with an accuracy of 77.3% (51.3% improvement over baseline; F1-score: 0.77), severe fatigue with an accuracy of 73.8% (45.0% improvement over baseline; F1-score: 0.74), and poor sleep quality with an accuracy of 72.0% (28.1% improvement over baseline; F1-score: 0.70). Further, sensor data were largely sufficient for predicting symptom severity, while the prediction of depressive symptoms benefited from minimal active patient input in the form of response to two brief questions on the day before the prediction point. ConclusionsOur digital phenotyping approach using passive sensors on smartphones and fitness trackers may help patients with real-world, continuous, self-monitoring of common symptoms in their own environment and assist clinicians with better triage of patient needs for timely interventions in MS (and potentially other chronic neurological disorders).

ImportanceArtificial Intelligence-driven analysis of laparoscopic video holds potential to increase the safety and precision of minimally invasive surgery. Vision-language models are particularly promising for video-based surgical decision support due to their capabilities to comprehend complex temporospatial (video) data. However, the same multimodal interfaces that enable such capabilities also introduce new vulnerabilities to manipulations through embedded deceptive text or images (prompt injection attacks). ObjectiveTo systematically evaluate how susceptible state-of-the-art video-capable vision-language models are to textual and visual prompt injection attacks in the context of clinically relevant surgical decision support tasks. Design, Setting, and ParticipantsIn this observational study, we systematically evaluated four state-of-the-art vision-language models, Gemini 1.5 Pro, Gemini 2.5 Pro, GPT-o4-mini-high, and Qwen 2.5-VL, across eleven surgical decision support tasks: detection of bleeding events, foreign objects, image distortions, critical view of safety assessment, and surgical skill assessment. Prompt injection scenarios involved misleading textual prompts and visual perturbations, displayed as white text overlay, applied at varying durations. Main Outcomes and MeasuresThe primary measure was model accuracy, contrasted between baseline performance and each prompt injection condition. ResultsAll vision-language models demonstrated good baseline accuracy, with Gemini 2.5 Pro generally achieving the highest mean [standard deviation] accuracy across all tasks (0.82 [0.01]), compared to Gemini 1.5 Pro (0.70 [0.03]) and GPT-o4 mini-high (0.67 [0.06]). Across tasks, Qwen 2.5-VL censored most outputs and achieved an accuracy of (0.58 [0.03]) on non-censored outputs. Textual and temporally-varying visual prompt injections reduced the accuracy for all models. Prolonged visual prompt injections were generally more harmful than single-frame injections. Gemini 2.5 Pro showed the greatest robustness and maintained stable performance for several tasks despite prompt injections, whereas GPT-o4-mini-high exhibited the highest vulnerability, with mean (standard deviation) accuracy across all tasks declining from 0.67 (0.06) at baseline to 0.24 (0.04) under full-duration visual prompt injection (P < .001). Conclusion and RelevanceThese findings indicate the critical need for robust temporal reasoning capabilities and specialized guardrails before vision-language models can be safely deployed for real-time surgical decision support. Key PointsO_ST_ABSQuestionC_ST_ABSAre video vision-language models (VLMs) susceptible to textual and visual prompt injection attacks when used for surgical decision support tasks? FindingTextual and visual prompt injection attacks consistently degraded the performance of four state-of-the-art VLMs across eleven surgical tasks. Gemini 2.5 Pro was most robust to textual and visual prompt injection attacks, whereas GPT-o4-mini-high was most vulnerable. Prolonged visual injections had a greater negative impact than single-frame injections. MeaningPresent-generation video VLMs are highly vulnerable to textual and visual prompt injection attacks. This critical safety vulnerability must be addressed before their integration into surgical decision support systems.

BackgroundProgress in artificial intelligence-based analysis of surgical videos has been constrained by reliance on manual frame-level annotations rather than patient-level outcomes. In addition, concerns about data privacy restrict the exchange of laparoscopic video data and, thereby, multicenter collaboration. MethodsTo address these limitations, we developed a pipeline that integrates weakly supervised deep learning with Swarm Learning, a decentralized machine learning approach that enables collaborative model training without data centralization. We evaluate our pipeline using a newly curated dataset of 397 laparoscopic appendectomy recordings from six international surgical centers. We identified optimal modelling configurations (frame sampling rates and model architectures) and subsequently compared Swarm Learning to single-center and centralized learning across three novel patient-level disease staging tasks: (i) binary detection of perforated appendicitis, (ii) laparoscopic grading of appendicitis, and (iii) histopathologic inflammation grading. In addition, we surveyed participating centers to identify real-world barriers to the clinical implementation of our decentralized learning pipeline for surgical video analysis. ResultsFor appendicitis grading tasks, frame sampling at 1.0 frames per second and use of the SurgTempoNet architecture resulted in reliable classification performance, outperforming SurgFrameNet and Multiple Instance Learning. Across all three disease staging tasks, Swarm Learning consistently outperformed single-center training and achieved performance comparable to centralized learning, with stable generalization in external validation. The user survey identified hardware failure and limited integration of the decentralized learning pipeline with electronic patient records as key barriers to the clinical implementation of our decentralized learning pipeline for collaborative surgical video analysis. ConclusionsWeakly supervised deep learning enables the prediction of patient-level endpoints directly from surgical video data. Swarm Learning facilitates privacy-preserving multicenter collaboration and achieves performance on par with centralized learning, highlighting its potential for advancing clinically relevant, collaborative AI development in surgical video analysis, especially when integrated with patients electronic health records. Article DescriptionThis study introduces a decentralized, privacy-preserving pipeline that combines weakly supervised deep learning with Swarm Learning to predict patient-level outcomes from laparoscopic appendectomy videos. Using data from six international surgical centers, the approach demonstrated performance comparable to centralized learning across three disease staging tasks while preserving data confidentiality by design.

Clinical foundation models learn patterns in longitudinal health records that can be reused across prediction tasks. However, the relative strengths of representation-based and generation-based prediction across clinical tasks remain unclear. Using longitudinal electronic health records from 3.77 million patients at a single hospital, we developed FERMAT (Foundation model for Exploring Real-world Multimodal health data using Autoregressive Trajectory modeling), a 65-million-parameter clinical foundation model. We compared predictions from task-specific predictors trained on patient representations with those derived from generated clinical event sequences. Representation-based prediction performed better in the disease-risk comparisons. In detailed analyses of five-year chronic kidney disease, fatty liver, and diabetes risk in a test cohort of 10,000 patients, representation-based predictors exceeded generation-based estimates by 0.098-0.115 in observation-weighted AUROC and had lower probability error. Generation-based prediction, in turn, performed better in predicting diagnoses, prescriptions, and procedures recorded on the same day, a task motivated by the frequent presence of multiple events with the same date in hospital EHRs. Using 50 generated trajectories per patient, it achieved a Recall@10 of 71.0%, compared with 55.7% for representation-based prediction, with a paired improvement of 15.31 percentage points (95% CI, 13.92-16.79; n = 1,862). These findings demonstrate task-dependent strengths of representation-based and generation-based applications within a single clinical foundation model, providing an empirical basis for exploring how the two approaches can be used individually or combined to address clinical prediction needs.

Large language models are increasingly deployed in clinical decision-support contexts, yet systematic evaluation of their factual reliability in generating patient-specific diagnostic reports remains sparse, particularly for laboratory interpretation tasks. This study presents a controlled transparency experiment in which four frontier LLMs -- Claude Sonnet 4.6, Claude Opus 4.6, GPT-5.2, and Gemini 3.1 Pro -- each generated diagnostic reports for 36 patients (29 female, 7 male; aged 27-64) with biomarker profiles spanning metabolic, endocrine, and nutritional markers. A transparency engine1 extracted up to 50 claims per report (3,035 total), searched for supporting scientific evidence, and classified each claim as supported by science, plausible, or unsupported. Unsupported claims were uncommon: the transparency engine classified 2.7% of claims as unsupported (hereafter, the pipeline-measured hallucination rate; naive claim-level 95% Wilson CI: 2.2%-3.4%), with GPT-5.2 at the lowest observed rate (1.7%) and Claude Opus 4.6 at the highest (3.6%). However, mechanistic verification revealed a much larger plausibility gap: 915 claims (30.2%) were biologically reasonable but lacked a fully verified evidence chain, bringing the share of claims not fully supported by direct evidence to 32.9%. Gemini 3.1 Pro produced the highest plausible proportion (39.6%), suggesting a more conservative but less fully grounded reasoning profile. Although coarse support-level distributions were broadly similar across models (Cramers V = 0.081), claim-level analysis revealed substantial narrative divergence: 61.2% of claims were unique to a single model, and matched-claim agreement was low (Cohens kappa = 0.233), indicating that models generate substantively different clinical narratives for the same patient data despite comparable aggregate support profiles. These findings show that hallucination metrics alone understate the share of claims not fully verified under this protocol, and that claim-level mechanistic verification is needed to distinguish the proven from the merely plausible in metabolic and endocrine laboratory interpretation, with generalizability to other clinical domains requiring further study.

ObjectiveLarge language models (LLMs) are increasingly embedded in mental-health chatbots, yet safe deployment is limited by two unresolved challenges: (1) suicide- and crisis-risk detection lacks a definitive ground truth and is characterized by substantial clinician disagreement, and (2) most evaluations frame risk detection as an offline accuracy task rather than a real-time safety problem. This study aimed to empirically characterize these limitations and to derive design principles for uncertainty-aware, safety-oriented crisis detection in conversational artificial intelligence. Methods and AnalysisWe curated a clinician-labeled dataset of 200 real-world conversation segments drawn from a deployed mental-health chatbot. Five clinical experts independently annotated each segment for suicide- and crisis-related risk. Using a single base LLM, we implemented five prompt-defined detection variants with systematically increasing sensitivity thresholds, without task-specific training or fine-tuning. Models were evaluated against clinician consensus labels to quantify false-negative and false-positive trade-offs. Latency analyses assessed feasibility for real-time, per-turn monitoring. ResultsAs sensitivity increased, the false-negative rate decreased monotonically from 87% to 0%, while false-positive rates rose accordingly. High- and extreme-sensitivity variants achieved near-perfect (98.9%) and perfect (100%) recall, demonstrating that near-zero-miss crisis detection from natural language is technically feasible in real time (mean latency <1 s). Importantly, model errors aligned closely with cases of clinician disagreement, indicating that misclassifications predominantly reflect irreducible uncertainty rather than model failure. ConclusionSuicide- and crisis-risk detection in conversational systems is inherently uncertain and should be reframed from an accuracy-oriented classification task toward an online, safety-oriented monitoring problem. Within this framing, near-zero-miss detection is achievable but necessarily incurs elevated false-positives, motivating architectural rather than purely model-level solutions. We propose an operational emergency mode in which conservative risk detection operates independently from the conversational model, allowing supportive engagement to be maintained under heightened safety constraints. This layered, uncertainty-aware architecture provides a practical pathway for safer deployment of LLM-based mental-health chatbots without reliance on large training datasets or extensive model optimization. What is already known on this topic?O_LIMental-health chatbots based on large language models are increasingly used for psychological support, but evaluations and real-world incidents show that general-purpose LLMs are unreliable in recognizing suicide- and crisis-related risk and may respond unsafely in high-stakes situations. C_LIO_LISuicide and crisis risk assessment lacks a stable ground truth, with substantial inter-clinician disagreement even among trained experts, indicating that automated risk detection is inherently uncertain and cannot be treated as a conventional supervised classification task. C_LI What this study addsO_LIDemonstrates that near-zero-miss suicide- and crisis-risk detection is technically feasible using clinician-validated data and prompt-based sensitivity calibration, without the further need for fine-tuning or large task-specific training datasets. C_LIO_LIShows that detection errors are largely driven by irreducible uncertainty rather than model failure, as misclassifications systematically align with areas of clinician disagreement, supporting the view that crisis detection is an online monitoring problem rather than a solvable classification task. C_LIO_LIIntroduces an architectural safety framework for mental-health chatbots in which conservative, independent risk detection enables an operational emergency mode that prioritizes safety while maintaining empathic engagement. C_LI How this study might afect research, practice or policyO_LIThis work reframes suicide- and crisis-risk detection in conversational AI as a safety-oriented, uncertainty-aware problem rather than an accuracy-driven prediction task, challenging prevailing evaluation practices. C_LIO_LIBy proposing an architectural separation between risk detection and dialogue generation, it provides a practical, scalable framework for deploying mental-health chatbots that support just-in-time safety interventions without relying on extensive clinical training data. C_LI

Digital phenotyping, which is defined as quantifying someone's behaviour with digital devices, provides unprecedented opportunities for understanding human mental health but is hampered by high levels of inter-individual variability. Here, we propose a new method to address this, parsing inter-individual variability by decomposing the digital phenotype dynamics into latent trajectories and using each individual's trajectory membership as a moderator when modelling psychopathology over the same timeframe. We applied our method in the context of mood symptom exacerbation across the menstrual cycle, where symptom severity and timing are inconsistent between individuals. Using the BiAffect platform to collect smartphone typing dynamics, we found stable trajectories in smartphone movement rate: one group of participants showed substantial movement rate fluctuations across the menstrual cycle, whilst the others did not. Participants with movement fluctuations displayed increased fluctuations across the cycle in prospective anhedonia and depression ratings, but not in anxiety, irritability, and suicidal ideation.

Importance. Current clinical artificial intelligence is bottlenecked by highly specialized, fragmented models that reduce complex, multimodal data into isolated scalar predictions. By reframing multimodal perception as a pure vision problem, optical compression into a universal visual operating system can resolve internal data interaction bottlenecks and shift the paradigm toward continuous, generative clinical forecasting. Objective. To develop and internally validate Clinical Visual Memory, a generalist visual foundation model utilizing high-fidelity optical compression to fuse five heterogeneous intensive care unit (ICU) data modalities into a unified visual-token space, acting as a generative navigation system for critical-care trajectories. Design, Setting, and Participants. Retrospective modeling study using MIMIC-IV adult ICU encounters. After excluding stays shorter than 24 hours, 54,551 patients, 68,546 hospital admissions, and 74,829 ICU stays comprised the eligible cohort. Strict patient-level partitioning prevented cross-split leakage. Methods. Structured electronic health record (EHR) data, vital signs, 10-second electrocardiogram (ECG) waveforms, chest radiographs, and clinical notes were rendered as 2D images and encoded by a single frozen DINOv2 vision transformer. Through modality-aware latent-query cross-attention, these highly heterogeneous sources were optically compressed into a shared 1024-dimensional Clinical Visual Memory. To establish a universal output interface, this memory conditioned an conditional instruction-tuned image generator to decode eight-domain deterioration trajectories across 3- to 48-hour horizons. Visual compression fidelity was evaluated by reducing retained source-pixel area to 1%, and critical transitions were mapped using event-specific projected-axis geometry. Results. Operating as a generalist foundation, Clinical Visual Memory achieved AUROCs of 0.852 (95% CI, 0.821-0.882) for 48-hour mortality, 0.699 (0.673-0.725) for incident acute kidney injury (AKI), 0.723 (0.688-0.757) for high Sequential Organ Failure Assessment (SOFA), and 0.742 (0.726-0.757) for alive ICU discharge. Crucially, resolving the token bottleneck via an 8-fold reduction in source-pixel area preserved 98.9% of the uncompressed 48-hour mortality AUROC. Generative image-out decoding maintained clinical meaning, and trajectory geometry functioned as a proactive navigation system, yielding median warning lead times of 34.8 hours (IQR, 17.5-43.3) before death, 14.6 hours (6.5-33.2) before AKI, 15.0 hours (6.0-27.0) before high SOFA, and 20.8 hours (10.2-36.0) before recovery, with low false-alert burdens (0.040-0.109 per patient-day). Conclusions and Relevance. Clinical Visual Memory demonstrates that optical compression can successfully unify highly heterogeneous clinical data into a single, high-fidelity visual-token representation. By functioning as a continuous clinical navigation system rather than a discrete alert generator, this framework lays the architectural foundation for a generalist, generative visual operating system in critical care medicine. External and prospective validation are required before clinical deployment.

Most clinical large language model (LLM) benchmarks rely on clean, concise vignettes that do not reflect the noisy, long-form documentation typical of real clinical records. How LLM performance degrades under realistic chart conditions remains poorly characterised. Here we test whether structured retrieval workflows protect National Institutes of Health Stroke Scale (NIHSS) scoring accuracy under systematic context stress. Using 100 de-identified acute stroke cases and a fully crossed 4 x 4 x 3 x 3 condition matrix (144 conditions per case), we vary context acquisition method, document length, distractor load and critical-information position across four Gemini models (57,047 retained runs). Structured retrieval reduces mean absolute error (MAE) from 4.58 to 2.96 points relative to non-agentic baselines (mean gain 1.62 MAE points; 95% CI 1.57 to 1.67; 35% relative reduction), with consistent gains across all 36 stress combinations. Lower-cost models show disproportionately larger gains (2.76 versus 0.45 MAE points). Tool-retrieved pipelines outperform retrieval-augmented generation in 33 of 36 combinations. These findings indicate that retrieval architecture, rather than model scale alone, is a tractable lever for robust, equitable clinical LLM deployment.

Structured AbstractO_ST_ABSBackgroundC_ST_ABSLarge language models (LLMs) are rapidly entering clinical care, yet their definitionally probabilistic outputs have delivered a variety of grossly unsafe responses to users. The difficulty in quantifying and mitigating the novel risks posed by LLMs threatens to stall the regulatory evaluation and clinical deployment of LLM-based software as a medical device (LLM-SaMD). A practical, evidence-based framework is urgently needed for extending existing medical-device regulations to encompass LLM-SaMDs. Using synthetic interactions between a chatbot and a potentially suicidal user, we demonstrate a simulation-based framework that provides a reproducible and generalizable method for evaluating the novel risks of LLM-SaMDs. MethodsWe developed a framework integrating LLM performance testing into SaMD risk estimation. Fourteen open-source models ranging from 270 million to 70 billion parameters (Qwen, Gemma, and LLaMA families) were evaluated on three safety-classification tasks: suicidal-ideation detection, therapy-request detection, and therapy-like interaction detection. Synthetic datasets were generated by Gemini 2.5 Pro and verified by psychiatrists. Model false-negative rates informed probabilistic estimates of P1, the likelihood of a hazard progressing to a hazardous situation, and P2, the likelihood of that situation resulting in harm. ResultsLLM success at generating synthetic safety datasets varied substantially by task, with strong performance for neutral and non-therapeutic content but frequent errors in suicidal-ideation and therapy-like interactions. Across 14 models (270 million-70 billion parameters), performance generally improved with size but included notable outliers. Estimated P1 values (hazard to hazardous situation) ranged from 2.0x10-8 to 2.6x10-4 and P2 (hazardous situation to harm) from 7.1x10-5 to 9.6x10-3, spanning up to four orders of magnitude. ConclusionSimulation extends existing device-safety frameworks to address the novel risks of large language models. Rather than replacing regulatory judgment, it provides a reproducible method for quantifying uncertainty, clarifying assumptions, and linking model failures to plausible harms. Our case example demonstrates a generalizable approach that can overcome current regulatory barriers while remaining practical for manufacturers and regulators, supporting timely and transparent oversight that keeps patients safe while avoiding unnecessary barriers to delivering the clinical promise of LLM-based medical devices. Brief DescriptionThis study introduces a quantitative framework for evaluating and mitigating the unique risks that large language models (LLMs) pose in healthcare. By mapping the pathways from LLM-generated hazards to harms onto existing regulatory risk-analysis structures and estimating the probability of these transitions through computational simulation, the framework empirically bounds uncertainty and identifies where real-world evidence is needed to validate and monitor model performance before, during, and after clinical deployment.

Surgery is inherently associated with complications, making early detection the cornerstone of timely intervention and improved outcomes. Artificial intelligence (AI) has been shown to predict severe events such as sepsis and mortality after surgery within intensive care units (ICUs). However, most complications occur on general wards (GW), where staffing and technical monitoring constraints impede effective real-time detection of complications. Here, we present a real-time, multimodal AI-based complication prediction system for surgical GWs, combining routine clinical data with continuous high-resolution vital signs derived from telemetric photopletysmography (PPG) sensors into digital patient representations. A total of 1,285 patients undergoing esophageal, gastric, liver, pancreatic, and colorectal surgery were prospectively enrolled. Baseline patient characteristics, intraoperative data, ICU parameters, and GW data, including 270,603 hours of recorded telemetric vital signs, were used to detect postoperative intra-abdominal infections. We demonstrate a high median area under the receiver operator characteristic (AUROC) of 0.90 (0.89-0.91) for the detection of surgery-related infections. Complications could be predicted 9 hours in advance with only a minor reduction of the AUROC: 0.89 (0.88-0.89). Continuous wearable data increased the AUROC by 8% and the Area Under the Precision-Recall Curve (AUPRC) by 109%, outperforming other modalities in our ablation experiments. Further development into an AI-based alarm system outperforms traditional early warning scores. These findings highlight the potential of high-dimensional, multimodal, real-time risk stratification to support earlier detection of adverse events in surgical patients. Our results reveal continuous monitoring, using minimally intrusive vital signs, as a key component of an intelligent, data-driven smart ward.

Systems powered by large language models are widely used for health information and advice, yet robust evidence for their safety and effectiveness in real-world clinical care remains lacking. Most existing studies evaluate general-purpose chatbots in artificial settings, failing to account for the critical role of system design, deployment context, and integrated safety mechanisms. Here, we report, to our knowledge, the first large-scale, clinician-blinded, real-world evaluation of a multi-agent LLM-based system deployed within a nationwide U.S. primary care telemedicine platform, assessing readiness for task-specific autonomous deployment. In 2,379 real patient encounters, where users actively sought medical care and completed full visits with licensed clinicians, we compared the AI system's intake diagnoses and disposition suggestions to those of treating clinicians, who were blinded to the AI's outputs. The AI's top-1 diagnosis matched the clinician's diagnosis in 91.3% of cases overall, increasing to 96.3% among cases meeting a pre-specified safety confidence threshold, and 97.9% in common, lower-complexity conditions that met the same confidence threshold. Disposition accuracy was similarly high, with an overall error rate of 2.5% and no errors in suggestions to emergency room or home management. These results demonstrate that purposeful system architecture, rather than model capability alone, is essential for safe and effective autonomous clinical AI. We propose a staged, task-calibrated deployment framework, in which AI can be introduced autonomously for well-defined tasks with explicit safety gating and continuous monitoring, expanding scope as real-world evidence accrues. Our findings provide the first real-world evidence of readiness for safe autonomous clinical AI and offer a practical roadmap for its responsible deployment at scale.

Artificial intelligence (AI) is increasingly permeating healthcare, from physician assistants to consumer applications. Since AI algorithms opacity challenges human interaction, explainable AI (XAI) addresses this by providing AI decision-making insight, but evidence suggests XAI can paradoxically induce over-reliance or bias. We present results from two large-scale experiments (623 lay people; 153 primary care physicians, PCPs) combining a fairness-based diagnosis AI model and different XAI explanations to examine how XAI assistance, particularly multimodal large language models (LLMs), influences diagnostic performance. AI assistance balanced across skin tones improved accuracy and reduced diagnostic disparities. However, LLM explanations yielded divergent effects: lay users showed higher automation bias - accuracy boosted when AI was correct, reduced when AI erred - while experienced PCPs remained resilient, benefiting irrespective of AI accuracy. Presenting AI suggestions first also led to worse outcomes when the AI was incorrect for both groups. These findings highlight XAIs varying impact based on expertise and timing, underscoring LLMs as a "double-edged sword" in medical AI and informing future human-AI collaborative system design.

There is great interest in personalized medicine, in which treatment is tailored to the individual characteristics of patients. Achieving the objectives of precision healthcare will require clinically-grounded, evidence-based approaches, which in turn demands rigorous, scalable predictive analytics. Standard strategies for deriving prediction models for medicine involve acquiring training data for large numbers of patients, labeling each patient according to the outcome of interest, and then using the labeled examples to learn to predict the outcome for new patients. Unfortunately, labeling individuals is time-consuming and expertise-intensive in medical applications and thus represents a major impediment to practical personalized medicine. We overcome this obstacle with a novel machine learning algorithm that enables individual-level prediction models to be induced from aggregate-level labeled data, which is readily-available in many health domains. The utility of the proposed learning methodology is demonstrated by: i.) leveraging US county-level mental health statistics to create a screening tool which detects individuals suffering from depression based upon their Twitter activity; ii.) designing a decision-support system that exploits aggregate clinical trials data on multiple sclerosis (MS) treatment to predict which therapy would work best for the presenting patient; iii.) employing group-level clinical trials data to induce a model able to find those MS patients likely to be helped by an experimental therapy.

Digital-medicine systems can objectively confirm medication ingestion, but interpretation of a no-detection day depends on whether the wearable receiver was in contact with the body. We analyzed 997 person-days from 20 outpatients prescribed aripiprazole through Abilify MyCite by integrating ingestion records with patch-contact data; 17 participants also contributed smartphone sensing. Of 322 days without a detected ingestion, 53.7% had insufficient patch wear. Counting all no-detection days as nonadherent yielded an apparent nonadherence rate of 32.3%, whereas conditioning on adequate wear yielded 18.1%. Because adherence during patch-off periods is unobserved, the rate is bounded between 14.9% and 32.3%. These findings support a three-state outcome that distinguishes detected ingestion, no ingestion despite adequate wear, and patch-off or indeterminate days. Among 744 patch-worn days, detected ingestion was more likely on days that more closely matched each participant's usual activity pattern (odds ratio 1.52, 95% credible interval 1.14-2.03) and when participants spent more time at home around the habitual dose period (1.31, 1.04-1.64). These behavioral associations were exploratory and characterize the context of medication-taking rather than causal effects. Overall, device wear was integral to interpretation of the adherence outcome, and passive behavioral measures were most interpretable after observation status was established.

Objective To develop and evaluate a framework for human-AI interaction. This approach, SHARE (Synergistic Human-Agent REasoning system) was designed to support scalable phenotyping of complex outcomes accurately, robustly and reproducibly from real-world electronic health record (EHR) data to support real-world evidence (RWE) generation. Methods and Analysis Using rheumatoid arthritis (RA) disease activity as the use-case, we studied a multi-institutional EHR-based RA cohort of 3,167 patients. Expert reviewers and a disease activity agent labeled notes using the same review guideline. The agent combined embedding-based informative-note filtering, structured evidence extraction, and evidence-based integrated reasoning to assign disease activity categories with supporting evidence, rationale, confidence, and ambiguity flags. To support scalable deployment, we evaluated a budget-tiered configuration using GPT-5 Nano for high-volume evidence extraction, o4-mini for final reasoning, benchmarking against a GPT-5.4 high reasoning effort configuration applied at every step. Note-level discrepancies were adjudicated by reviewers into final co-produced labels that were used to refine labels and inform agent development. The main outcome measure was the mean absolute error (MAE) of the initial and final agent vs the final co-produced labels. The agreement between agent- and reviewer-flagged ambiguous notes, per-note cost and compute time across configurations were also tested. Results Expert reviewers labeled 626 notes from 273 patients; human-AI adjudication revised 127 (20%) of these initial labels and added 60 newly labeled notes, yielding a 686-note co-produced reference. Against this reference, the final agent's accuracy improved from a mean absolute error of 0.406 to 0.291 with co-learning, and its ambiguity flag agreed with expert ambiguity designations with 92.1% accuracy. Applied across the cohort, the agent labeled 101,691 notes; the budget tiered configuration matched the accuracy of GPT-5.4 at high reasoning effort while reducing estimated cost by 69% and compute time by 70%. Conclusion Adopting a framework for human-AI co-learning, SHARE, improved the overall quality of gold-standard labels, identified ambiguous cases for further review, and supported accurate and standardized chart reviews of disease activity at a scale infeasible for manual review. SHARE's resource efficiency provides a transferable approach to incorporate complex phenotypes in RWE studies.

Forecasting a patient's laboratory measurements at future clinical visits from longitudinal electronic health records (EHRs) can support disease monitoring and treatment planning in the context of personalized medicine. However, accurate prediction remains challenging since patients exhibit complex and highly individualized clinical trajectories. Here, we present LaBERT, a transformer-based model trained to forecast future laboratory measurements of a patient given information available at the current and previous clinical visits. Evaluated on 583,535 clinical visits from 255,769 patients in the MIMIC-IV database, LaBERT consistently outperformed baseline methods, reducing mean squared error from 0.77 to 0.53 and improving the coefficient of determination (R2) from 0.29 to 0.51. Medication perturbation analysis further showed that LaBERT learns treatment-related information that is clinically meaningful. In particular, we showed that using the originally prescribed medications, LaBERT predicted future patient states more accurately than when using randomized medication sets in 81% of visits. Furthermore, our controlled counterfactual analyses reproduced established pharmacological effects, including warfarin-associated increases in international normalized ratio (INR) and heparin-associated increases in activated partial thromboplastin time (aPTT), consistently across multiple prediction horizons. These findings establish LaBERT as a model for forecasting future laboratory measurements from longitudinal EHRs and provide a foundation for treatment-dependent patient-state simulation and personalized clinical decision support.

Objective: Online health information seeking is rising, and individuals increasingly act on peer advice without clinical oversight, adjusting doses, delaying care, and modifying treatment. Current misinformation detection assumes factually inaccurate content is what makes these decisions unsafe. We introduce VERITAS (Verification Engine for Risk-aware Information Trust Assessment in health Stories) and formalize the Risk Irrelevance Principle: divergence from accepted clinical practice and potential for harm are distinct, weakly associated dimensions that must be assessed separately. Materials and Methods: VERITAS transforms unstructured health narratives into Agent-Action-Outcome graphs and computes two continuous metrics: Narrative Truth Distance (NTD), quantifying epistemic divergence, and Narrative Risk Score (NRS), assessing harm potential. We evaluated VERITAS on 704 threads from four Reddit health communities. Two domain experts annotated 2,000 segments (Krippendorffs =0.78-0.81). NTD-NRS independence was validated using seven tests. Results: NTD and NRS shared under 5% of variance (r = 0.222; mutual information 0.096 bits): a posts divergence from consensus conveys little about whether acting on it will cause harm. On 435 labeled posts, VERITAS identified 62.2% of expert-labeled misinformation versus 57.5% for the strongest text classifier, the gain concentrated in factually plausible content describing unsafe self-management (27.6% of misinformation) that accuracy-focused classifiers approve. VERITAS assessed 37.8% of this misinformation as low-risk, pending clinical validation. Discussion: Fact-checking-based screening systematically approves the content most likely to prompt unsafe self-management while flagging content least likely to cause harm. Conclusion: Separating divergence from harm potential shifts verification from whether information is correct to whether it is safe to act upon.

Objective: Predicting health outcomes from electronic health records (EHRs) is challenging because traditional models rely on structured data and often ignore external medical knowledge. We propose an approach that integrates structured EHR with text-based clinical evidence to improve prediction and interpretability. Methods: We introduce PHO-Agents, a multi-agent system powered by large language models (LLMs) for health outcome prediction. Structured EHR sequences are encoded to produce attention based representations and initial logits, which are converted into patient summaries by a data agent. A retrieval agent gathers relevant clinical guidelines. Research and practical doctor agents independently assess the patient, and a leader agent synthesizes their analyses. Outputs from the EHR based model and the LLM agents are fused to generate final predictions and explanation reports. PHO-Agents was evaluated on three real-world cohorts: acute kidney injury (AKI) patients (in-hospital mortality), chronic kidney disease patients (AKI onset within two years), and cancer patients receiving immune checkpoint inhibitors (immune-related adverse events within one year). Results: PHO-Agents outperformed single-agent and multi-agent LLM baselines across all cohorts. In the AKI mortality task, it achieved a PR-AUC of 90.20 +/- 2.07, compared with 56.46 +/- 2.98 for the best single-agent baseline. Similar gains were observed in the ICI and CKD cohorts. Ablation studies showed that both multi-agent reasoning and logit-level fusion contributed to performance improvements, and case analyses demonstrated clinically consistent explanations. Conclusion: PHO-Agents integrates longitudinal EHR modeling with collaborative LLM reasoning, improving predictive performance, interpretability, and robustness across diverse clinical tasks. This hybrid approach offers a trustworthy strategy for real-world clinical decision support.

The widespread adoption of clinical large language models (LLMs) introduces significant risks of automation bias, premature closure, and clinician deskilling. Current interpretability paradigms, including latent space trajectories, Concept Activation Vectors, and Concept Bottleneck Models, suffer from topological stagnation, metric distortion, and epistemic occlusion, frequently masking intermediate diagnostic uncertainty behind falsely confident outputs. To address these structural vulnerabilities, this paper introduces a novel closed-loop, multi-agent framework designed to quantify and visualize dynamic epistemic uncertainty in clinical LLM reasoning. By coupling predictive Shannon entropy with non-linear Isometric Feature Mapping (ISOMAP), the architecture projects high-dimensional inference state vectors onto a calibrated two-dimensional latent space, thereby assigning a quantifiable thermodynamic energy state to the reasoning path to track diagnostic velocity, cognitive momentum, and trajectory efficiency across sequential diagnostic rounds. Pilot validation across representative emergency medicine scenarios demonstrated distinct topological and information-theoretic behaviors: unconfounded cases (cerebellar infarction) exhibited smooth geodesic progression toward the ground truth alongside monotonic Shannon entropy decay from 2.15 to 1.74; noisy environments with ambiguous findings (spontaneous pneumothorax) suffered from trajectory wandering, local minimum traps, and high sustained entropy (~2.41) due to insufficient repulsive weighting for negative evidence; and triage-conflicted cases (acute cholangitis) achieved precise geometric proximity to the true node but experienced top-1 rank stagnation because the model conflated acute severity triage (sepsis) with anatomical etiology. By rendering machine hesitation and cognitive divergence visually auditable before final diagnostic crystallization, this geometric-information framework enables dynamic trust calibration and human-AI co-regulation at the point of care while establishing a clear mathematical foundation for future architectural interventions, such as dual-channel safety decoupling and non-linear repulsive weighting. Moving forward, validating these architectural enhancements across large-scale electronic health record databases and prospective clinical trials will be essential to realize its full clinical utility, establishing a foundational blueprint for safe, transparent, and cognitively synergistic AI integration in future medical practice. By rendering the LLM's reasoning process visually auditable, this framework lays the groundwork for capturing and externalizing the clinician's own cognitive patterns within the AI, forming a coupled system. This enables the explicit visualization of cognitive gaps between physician hypotheses and AI inferences, transforming the interaction from simple answer-checking into a dynamic learning process for both human and machine that prevents diagnostic oversight. Ultimately, because the responsibility for final clinical decision-making remains with the human practitioner, this framework serves as a vital decision-support mechanism. Moving forward, validating these architectural enhancements across large-scale electronic health record databases and prospective clinical trials will be essential to realize its full clinical utility, establishing a foundational blueprint for safe, transparent, and cognitively synergistic AI integration in future medical practice.